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ABSTRACT 

This  paper  describes  a  vector  space  approach  to  solving  the 
multidimensional  (to- D)  Yule- Walker  equations  for  an  ar¬ 
bitrary  region  of  support.  This  approach  leads  to  a  solution 
that  is  simple  to  implement. 

1.  INTRODUCTION 

Two-dimensional  (2-D)  autoregressive  (AR)  modelling  has 
found  applications  in  image  processing,  sonar,  and  other  ar¬ 
eas.  3-D  and  higher  dimensional  AR  modelling  has  as  yet  to 
be  extensively  studied  but  applications  to  signals  that  vary 
in  time,  frequency,  and  space  can  easily  be  envisioned. 

2.  PROBLEM  STATEMENT 


3.  SOLVING  THE  MULTIDIMENSIONAL 
YULE-WALKER  EQUATIONS 

Consider  an  to- D  AR  process. 

x[n]  =  —  ^  a[k]x[n  —  k]  +  u[n]  (1) 

keSm 

where  n  =  [n i  ri2  ...  nm]T ,  S'm  is  the  region  of  support 
for  a[k],  and  u[n]  is  white  noise.  The  m- D  Yule  Walker 
equations  are  found  as  follows 

E[a;[n]a;*[n  —  1]]  = 

—  ^  a[k]E[a;[n  —  k]x*[n  —  1]]  +  E[u[n]a;*[n  —  1]].  (2) 

kes;. 

Defining  the  autocorrelation  function  as 


The  Yule- Walker  equations  in  one  dimension  can  be  very 
conveniently  represented  in  matrix  format  (see  Kay  [1]). 
The  matrix  approach  works  well  because  the  region  of  sup¬ 
port  (ROS)  for  the  filter  parameters  of  a  1-D  AR  process  is  a 
line  segment,  which  leads  to  a  set  of  Yule- Walker  equations 
that  can  easily  be  put  into  a  matrix  format.  This  convenient 
representation  leads  to  relatively  simple  algorithms  that  are 
easily  implemented  in  MATLAB  or  C. 

2-D  AR  models,  on  the  other  hand,  can  be  put  into  a  ma¬ 
trix  format  but  the  representation  is  less  natural  and  more 
forced.  This  is  caused  by  the  2-D  ROS,  which  leads  to  a 
much  more  complicated  set  of  linear  equations  [1],[2].  Less 
insight  is  gained  from  the  matrix  format  and  coding  is  more 
difficult.  Forcing  3-D  and  higher  dimensional  models  into 
a  matrix  format  is  even  more  strained  and  leads  to  an  ex¬ 
tremely  difficult  implementation. 

The  motivation  in  using  a  vector  space  approach  is  that 
it  leads  to  a  natural  method  of  representing  the  to-D  Yule- 
Walker  equations.  This  is  because  vector  spaces  easily  gen¬ 
eralize  to  higher  dimensions.  The  vector  space  method  also 
aids  one’s  intuition  in  developing  and  coding  algorithms. 


rx  [k]  =  S[x*[n]a’[n  +  k]] 
we  have  from  (2)  that 

r*M  =  -  53  ^kKP-k]  les;.  (3) 
kes;. 

This  assumes  that  E[ii[n]a;*[n  —  1]]  =0  for  1  £  S'm.  Note 
from  (1)  that  x[n  —  1]  for  1  £  S'm  constitutes  the  ’’past”, 
which  is  uncorrelated  with  tt[n].  Also  from  (1) 


2" 

E[Mn]|2]=E 

«[kMn  -  k] 

kesm 

where  Sm  =  S'm  (J{k  =  0}  and  a[0]  =  1.  This  can  be 
shown  to  reduce  to 


=  E 


a[k].c[n  —  k]x*  [n] 

-kesm 


=  a[k]rx[-k], 

k  (zSm 


(5) 


Report  Documentation  Page 

Form  Approved 

OMB  No.  0704-0188 

Public  reporting  burden  for  the  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering  and 
maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information, 
including  suggestions  for  reducing  this  burden,  to  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports,  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington 

VA  22202-4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  a  penalty  for  failing  to  comply  with  a  collection  of  information  if  it 
does  not  display  a  currently  valid  OMB  control  number. 

1 .  REPORT  DATE  2.  REPORT  TYPE 

2003  N/A 

3.  DATES  COVERED 

4.  TITLE  AND  SUBTITLE 

Vector  Space  Solution  To  The  Multidimensional  Yule-Walker  Equations 

5a.  CONTRACT  NUMBER 

5b.  GRANT  NUMBER 

5c.  PROGRAM  ELEMENT  NUMBER 

6.  AUTHOR(S) 

5d.  PROJECT  NUMBER 

5e.  TASK  NUMBER 

5f.  WORK  UNIT  NUMBER 

7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES) 

Naval  Undersea  Warfare  Center  Newport,  R1 02841 

8.  PERFORMING  ORGANIZATION 

REPORT  NUMBER 

9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES) 

10.  SPONSOR/MONITOR'S  ACRONYM(S) 

11.  SPONSOR/MONITOR'S  REPORT 
NUMBER(S) 

12.  DISTRIBUTION/AVAILABILITY  STATEMENT 

Approved  for  public  release,  distribution  unlimited 

13.  SUPPLEMENTARY  NOTES 

14.  ABSTRACT 

15.  SUBJECT  TERMS 

16.  SECURITY  CLASSIFICATION  OF:  17.  LIMITATION  OF 

18.  NUMBER  19a.  NAME  OF 

a.  REPORT  b.  ABSTRACT  c.  THIS  PAGE  |J|J 

unclassified  unclassified  unclassified 

4 

Standard  Form  298  (Rev.  8-98) 

Prescribed  by  ANSI  Std  Z39-18 


As  a  result  (3)  and  (5)  can  be  combined  to  yield  the  com¬ 
plete  set  of  Yule- Walker  equations 

E  a[k]rx[l  -  k]  =  cr2A[l]  for  1  G  Sm  (6) 
keSm 


where 


m 


1  1  =  0 

0  1^0. 


Letting  6[k]  =  a[k]/cr2,  this  becomes 


E  6[k]rx[l-k]=5[l]  \eSm.  (7) 

kGSm 

We  can  view  (7)  as  a  linear  transformation,  which  trans¬ 
forms  the  vector  6[k]  into  the  vector  <5[k] .  We  represent  this 
transformation  symbolically  as 

£{6[k]}  =  <5[k],  (8) 

Note  that  the  domain  and  range  spaces  are  the  same  since 
fo[k]  G  V,  h[k]  G  V  where  V  is  the  space  of  complex  se¬ 
quences  defined  on  Srn  .To  appreciate  the  generality  of  (8) 
we  give  a  few  examples.  In  Figure  la  we  show  the  usual 
ROS  for  the  parameters  of  a  1-D  AR  process  (a[0]  =  1 
is  included  as  it  is  for  the  following  examples).  In  Figure 
lb  the  ROS  is  shown  for  a  2-D  quarter  plane,  in  Figure  lc 
the  ROS  is  shown  for  a  2-D  nonsymmetric  half  plane  and 
finally,  in  Figure  Id  the  ROS  is  shown  for  a  3-D  nonsym¬ 
metric  half  space.  The  process  parameters  are  all  defined  by 
6[k]  =  a[k]/tr2  for  k  G  Sm. 


The  natural  basis  for  V  is  {ei,  e%,  ■  •  • ,  e\sm\}’  where  e,  has 
a  1  in  its  ith  position  and  0  otherwise.  To  solve  (8)  for  6[k] 
we  first  define  an  inner  product  as 

<v,w>=  E  E  “  k]wj*[!].  (9) 

kGSml6Sm 

To  verify  that  this  is  a  valid  inner  product  we  note  that  rx  [1  — 
k]  can  be  written  as 

rx[l  -  k]  =  £,[a;*[k]a’[l]]  (10) 

and  as  a  result 


<  v,w  >  = 


E 


E  E  **[k]z[lK[l] 

_kGSmleSm 


=  E 


_kesm  l  esm 


from  which  the  usual  properties  of  the  inner  product  follow. 
Assume  we  can  find  an  orthonormal  basis  for  V  or 

{i>i,  v2,  ■  ■  • ,  f|sm|}  where  <  v^,  Vj  >=  Sij  so  that 


EE*-  t1  -  kK  M  =  |  ?  f  J-  (n) 

keSm  iesm  l1  * 

Since  6[k]  G  V  we  can  represent  it  as  a  linear  combination 
of  basis  vectors,  b[ k]  =  E=l'  PivA k]  and  the  Yule- Walker 
equations  then  become  from  (7) 


Fig.  1.  Illustration  of  Common  Regions  of  Supports  for  AR 
Process  Parameters 

Now  consider  rasa  vector  in  V.  The  dimension  of  V 
is  equal  to  the  number  of  elements  in  Sm  and  is  denoted 
by  \Sm\.  For  example  from  Figure  la,b,c,d  we  have  that 
,S'i  =  5,  | S2|  =  6,  | S'2  =  13,  and  .S3  =  14,  respectively. 


ISA  I 

E  Pi  E  -  k]  =  S[\]  1  G  Sm.  (12) 

i=l  k  G5m 

Now  multiply  by  v*  [1]  for  each  1  G  Sm  and  sum  to  yield 

|Sm| 

EftEE  ui[k]rx[l  —  k]u*[l]  =  E  (13) 

i- 1  kesmiesm  iesm 

or 

|Sm| 

E  Pi  <  Vi ,  Vj  >  =  e  vj  (i4) 

7—1  lG^m 

But  <  Vi  ,  Vj  >  =  S^,  so  that 

Pj  =  E  wm  as) 

ieSm 

for  ji  =  1,2,...,  |Sm|.  Therefore, 

\sm\ 

Mk]  =  EE  <m^w«i[k]. 

»= 1  1GSA 


(16) 


Finally,  since  Ei<=s  =  vt  I®]-  t^le  solution  be¬ 

comes 

|Sm| 

b[  k]  =  $>*[0h[k].  (17) 

i= 1 

We  have  assumed  that  we  can  find  an  orthonormal  basis 
for  V.  To  obtain  this  basis  for  V  a  Gram-Schmidt  (GS)  or- 
thogonalization  is  performed  on  {ei,  e2,  . . . ,  e\sm\}  which 
produces  {tq,  v2,  . . . ,  U|sm|}.  It  proceeds  as  follows: 


ei 


and  for  k  =  2, 3, ... ,  |Sm| 

vk  =  e^^U<e^vi>Vi  (lg) 

|| ek  -  Ei=t  <  efc>  vi  >  ^11 

It  is  important  to  realize  that  no  matrix  inversions  are  re¬ 
quired.  The  principal  source  of  computation  is  due  to  the 
need  to  compute  inner  products.  However,  these  are  easily 
done  as  follows.  The  inner  products  can  be  computed  as 

<v,w>  =  ^  y;  v  [k]  rx  [1  —  k]  w*  [1] 

kesm  iesm 

=  X)  5Z  ^M^l1 -ki¬ 
te  sm  kGSm 

Let  z[l]  =  Ekesm  w[kK[l  -  k].  Then, 

<v,w  >=  y  u>*[i]^[i]. 
lGSm 

To  calculate  the  above  we  shift  the  rx  array  and  then  per¬ 
form  an  element  by  element  array  multiply  and  sum. 

4.  RESULTS 

We  will  go  through  two  examples  to  demonstrate  some  re¬ 
sults  and  suggest  how  to  use  MATLAB  to  solve  the  Yule- 
Walker  equations.  For  the  first  example,  we  will  use  the  sep¬ 
arable  3-D  AR(1,1,1)  process  with  QP  ROS  that  was  used 
by  Choi  (see  [3]) 

x[ni,ri2,  n3]  =  0.9x[ni  -  1,  n2,  n3]  +  0.88x[ni,  n2  —  1,  n3] 
+  0.95x[ni,  n2,  n3  —  1]  —  0.7920x[ni  —  l,n2  —  l,n3] 

—  0.8550x[rii  —  1,  n2,  n3  —  1]  —  0.8360x[ni,  n2  —  1,  n3  —  1] 
+  0.7524x[ni  -  1,  n2  —  1,  n3  —  1]  4-  u[ni,  n2,  n3]. 

The  autocorrelation  function  is 

rx[ki,k2,k3]  =  0.9|fel|0.88|fc2|0.95|fe31.  (19) 

To  use  the  theory  developed  in  this  paper  to  find  the  AR 
parameters  the  following  steps  must  be  coded. 


1 .  Determine  the  ROS  and  hence  indices  of  the  AR  pa¬ 
rameters  in  the  ROS. 

2.  With  the  indices  of  the  AR  parameters  determine  the 
correlations  needed  in  rx[  1  —  k]  (see  9). 

3.  Use  the  Gram-Schmidt  process  of  (18)  to  find  an  or¬ 
thonormal  basis  using  the  inner  product  in  (9). 

4.  The  AR  parameters  are  found  directly  from  (17). 

Using  the  above  we  obtain 

a[0, 0, 0]  =  1.0000 
a[l,  0, 0]  =  —0.9000 
a[0, 1, 0]  =  -0.8800 
a[0, 0, 1]  =  -0.9500 
a[l,  1,0]  =  0.7920 
a[l,0, 1]  =  0.8550 
a[0, 1, 1]  =  0.8360 
a[l,  1, 1]  =  -0.7524 

which  is  the  correct  result. 

For  the  next  example  we  use  another  3-D  AR(  1,1,1)  pro¬ 
cess  but  this  time  it  is  non-separable  and  has  the  NSHP  ROS 
as  in  Figure  Id. 

x[ni,n2,n3]  =0.80x[?ti  —  l,n2  +  l,n3  +  1] 

+  0.50x[ni  —  1,  n2  —  1,  n3  —  1]  +  u[m,  n2, 

The  vector  space  method  results  in 

a[0, 0, 0]  =  1.0000 
a[l,  —1,  — 1]  =  -0.8000 
a[l,  1, 1]  =  -0.5000 

the  rest  of  the  parameters  are  zero,  as  expected. 

5.  CONCLUSIONS 

In  this  paper  we  have  developed  a  simple  method  to  solve 
the  m-dimensional  Yule-Walker  equations  with  an  arbitrary 
region  of  support.  No  matrix  inversions  are  required  and 
coding  is  easily  accomplished.  A  complete  implementation 
in  MATLAB  is  available  upon  request. 
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